An Automatic Lip-reading Method Based on Polynomial Fitting

This paper addresses the problem of speaker-dependent isolate digits recognition using sole visual information. We employ intensity transformation and spatial filter to estimate the minimum enclosing rectangle of mouth in each frame. Thus, for each utterance, we can obtain two vectors composed of width and height of mouth, respectively. Then, we propose an approach to recognize the speech based on polynomial fitting. Firstly, both width and height vectors are normalized into the constant length via interpolation. Secondly, least square method is utilized to produce two 3order polynomials that can represent the main trend of the two vectors, respectively, and reduce the noise caused by the estimate error. Lastly, positions of three crucial points (i.e. maximum, minimum and right boundary point) in each 3-order polynomial curve are recorded as a feature vector. For each utterance, we calculate the average of all vectors of training sample to make a template, and using Euclidean distance between the template and testing data to perform the classification. Experiments show the promising results of the proposed approach.